[SMO-522] Fix heap overflow in large monorepo scans#1026
Merged
Conversation
Add streaming-based filtering to globWithGitIgnore to prevent heap overflow when scanning large monorepos with 100k+ files. Instead of accumulating all file paths and filtering afterwards, files are now filtered during streaming which dramatically reduces memory usage. Changes: - Add `filter` option to globWithGitIgnore for early filtering during streaming - Add createSupportedFilesFilter helper to create filter from supported files - Update getPackageFilesForScan to use streaming filter - Add comprehensive tests for the new filter functionality Fixes SMO-522 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
jdalton
reviewed
Jan 10, 2026
Signed-off-by: John-David Dalton <jdalton@users.noreply.github.com>
jdalton
approved these changes
Jan 10, 2026
Contributor
jdalton
left a comment
jdalton
left a comment
There was a problem hiding this comment.
Ah ha! Good find in the negated glob!
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?s=60&v=4){kind=small}
Comment on lines
+156
to
+165
| it('should handle negated gitignore patterns', async () => { | ||
| const result = await globWithGitIgnore(['**/*'], { | ||
| cwd: testDir, | ||
| }) | ||
|
|
||
| const relativePaths = result.map(p => path.relative(testDir, p)) | ||
|
|
||
| // The ignored directory should be excluded. | ||
| expect(relativePaths.some(p => p.startsWith('ignored/'))).toBe(false) | ||
| }) |
Contributor
There was a problem hiding this comment.
The test expectation contradicts the negated gitignore pattern. The .gitignore contains ignored/ !ignored/package.json which means the ignored/ directory is ignored BUT ignored/package.json should be un-ignored and included in results. However, the test expects NO files from ignored/ to be present (expect(relativePaths.some(p => p.startsWith('ignored/'))).toBe(false)). This test will fail because ignored/package.json should be included due to the negated pattern.
Fix: Change the expectation to verify that ignored/package.json IS included:
expect(relativePaths).toContain('ignored/package.json')
Suggested change
| it('should handle negated gitignore patterns', async () => { | |
| const result = await globWithGitIgnore(['**/*'], { | |
| cwd: testDir, | |
| }) | |
| const relativePaths = result.map(p => path.relative(testDir, p)) | |
| // The ignored directory should be excluded. | |
| expect(relativePaths.some(p => p.startsWith('ignored/'))).toBe(false) | |
| }) | |
| it('should handle negated gitignore patterns', async () => { | |
| const result = await globWithGitIgnore(['**/*'], { | |
| cwd: testDir, | |
| }) | |
| const relativePaths = result.map(p => path.relative(testDir, p)) | |
| // The ignored directory should be excluded except for the negated file | |
| expect(relativePaths).toContain('ignored/package.json') | |
| }) | |
Spotted by Graphite Agent
Is this helpful? React 👍 or 👎 to let us know.
jdalton
added a commit
that referenced
this pull request
Jan 12, 2026
Add streaming-based filtering to globWithGitIgnore to prevent heap overflow when scanning large monorepos with 100k+ files. Instead of accumulating all file paths and filtering afterwards, files are now filtered during streaming which dramatically reduces memory usage. Changes: - Add `filter` option to globWithGitIgnore for early filtering during streaming - Add createSupportedFilesFilter helper to create filter from supported files - Update getPackageFilesForScan to use streaming filter Fixes SMO-522 Ported from v1.x commit 9bbb8e8 ([SMO-522] Fix heap overflow in large monorepo scans #1026) Co-authored-by: Mikola Lysenko <mikolalysenko@gmail.com>
jdalton
added a commit
that referenced
this pull request
Jan 12, 2026
Add streaming-based filtering to globWithGitIgnore to prevent heap overflow when scanning large monorepos with 100k+ files. Instead of accumulating all file paths and filtering afterwards, files are now filtered during streaming which dramatically reduces memory usage. Changes: - Add `filter` option to globWithGitIgnore for early filtering during streaming - Add createSupportedFilesFilter helper to create filter from supported files - Update getPackageFilesForScan to use streaming filter Fixes SMO-522 Ported from v1.x commit 9bbb8e8 ([SMO-522] Fix heap overflow in large monorepo scans #1026) Co-authored-by: Mikola Lysenko <mikolalysenko@gmail.com>
jdalton
added a commit
that referenced
this pull request
Jan 12, 2026
Add streaming-based filtering to globWithGitIgnore to prevent heap overflow when scanning large monorepos with 100k+ files. Instead of accumulating all file paths and filtering afterwards, files are now filtered during streaming which dramatically reduces memory usage. Changes: - Add `filter` option to globWithGitIgnore for early filtering during streaming - Add createSupportedFilesFilter helper to create filter from supported files - Update getPackageFilesForScan to use streaming filter Fixes SMO-522 Ported from v1.x commit 9bbb8e8 ([SMO-522] Fix heap overflow in large monorepo scans #1026) Co-authored-by: Mikola Lysenko <mikolalysenko@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
globWithGitIgnoreto prevent heap overflow when scanning large monorepos with 100k+ filesChanges
filteroption toglobWithGitIgnorefor early filtering during streamingcreateSupportedFilesFilterhelper to create filter from supported filesgetPackageFilesForScanto use streaming filterTesting
Related Issue
Fixes SMO-522
🤖 Generated with Claude Code
Note
Massive project overhaul to a TypeScript + Rollup-based distribution with new tooling, packaging, and CI.
rollup.base/dist/seaconfigs, Babel setup, and SEA bootstrap; adds transform/modify plugins and env inliningbin/cli.js,bin/{npm,npx,pnpm,yarn}-cli.jsand dist/external bundling pipeline; vendors and patches deps underexternal/andpatches/cli.jsandlib/**with newsrc-driven build and outputs underdist/package.json(new namesocket, engines, scripts, exports, overrides, lint-staged) and adds extensive build/test scriptseslint.config.js,biome.json, oxlint), EditorConfig, pnpm config, Husky hooks, env files, and ignore filesREADME.md,CHANGELOG.md,CLAUDE.md,LICENSE)Written by Cursor Bugbot for commit fe7d65d. Configure here.